Artificial intelligence (AI) and machine learning (ML) in precision oncology: a review on enhancing discoverability through multiomics integration.

Multiomics data including imaging radiomics and various types of molecular biomarkers have been increasingly investigated for better diagnosis and therapy in the era of precision oncology. Artificial intelligence (AI) including machine learning (ML) and deep learning (DL) techniques combined with the exponential growth of multiomics data may have great potential to revolutionize cancer subtyping, risk stratification, prognostication, prediction and clinical decision-making. In this article, we first present different categories of multiomics data and their roles in diagnosis and therapy. Second, AI-based data fusion methods and modeling methods as well as different validation schemes are illustrated. Third, the applications and examples of multiomics research in oncology are demonstrated. Finally, the challenges regarding the heterogeneity data set, availability of omics data, and validation of the research are discussed. The transition of multiomics research to real clinics still requires consistent efforts in standardizing omics data collection and analysis, building computational infrastructure for data sharing and storing, developing advanced methods to improve data fusion and interpretability, and ultimately, conducting large-scale prospective clinical trials to fill the gap between study findings and clinical benefits.

Interpretable artificial intelligence in radiology and radiation oncology.

Artificial intelligence has been introduced to clinical practice, especially radiology and radiation oncology, from image segmentation, diagnosis, treatment planning and prognosis. It is not only crucial to have an accurate artificial intelligence model, but also to understand the internal logic and gain the trust of the experts. This review is intended to provide some insights into core concepts of the interpretability, the state-of-the-art methods for understanding the machine learning models, the evaluation of these methods, identifying some challenges and limits of them, and gives some examples of medical applications.

The Promise and Future of Radiomics for Personalized Radiotherapy Dosing and Adaptation.

Quantitative image analysis, also known as radiomics, aims to analyze large-scale quantitative features extracted from acquired medical images using hand-crafted or machine-engineered feature extraction approaches. Radiomics has great potential for a variety of clinical applications in radiation oncology, an image-rich treatment modality that utilizes computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) for treatment planning, dose calculation, and image guidance. A promising application of radiomics is in predicting treatment outcomes after radiotherapy such as local control and treatment-related toxicity using features extracted from pretreatment and on-treatment images. Based on these individualized predictions of treatment outcomes, radiotherapy dose can be sculpted to meet the specific needs and preferences of each patient. Radiomics can aid in tumor characterization for personalized targeting, especially for identifying high-risk regions within a tumor that cannot be easily discerned based on size or intensity alone. Radiomics-based treatment response prediction can aid in developing personalized fractionation and dose adjustments. In order to make radiomics models more applicable across different institutions with varying scanners and patient populations, further efforts are needed to harmonize and standardize the acquisition protocols by minimizing uncertainties within the imaging data.

Deep learning prediction of post-SBRT liver function changes and NTCP modeling in hepatocellular carcinoma based on DGAE-MRI.

BACKGROUND: Stereotactic body radiation therapy (SBRT) produces excellent local control for patients with hepatocellular carcinoma (HCC). However, the risk of toxicity for normal liver tissue is still a limiting factor. Normal tissue complication probability (NTCP) models have been proposed to estimate the toxicity with the assumption of uniform liver function distribution, which is not optimal. With more accurate regional liver functional imaging available for individual patient, we can improve the estimation and be more patient-specific. PURPOSE: To develop normal tissue complication probability (NTCP) models using pre-/during-treatment (RT) dynamic Gadoxetic Acid-enhanced (DGAE) MRI for adaptation of RT in a patient-specific manner in hepatocellular cancer (HCC) patients who receive SBRT. METHODS: 24 of 146 HCC patients who received SBRT underwent DGAE MRI. Physical doses were converted into EQD2 for analysis. Voxel-by-voxel quantification of the contrast uptake rate (k1) from DGAE-MRI was used to quantify liver function. A logistic dose-response model was used to estimate the fraction of liver functional loss, and NTCP was estimated using the cumulative functional reserve model for changes in Child-Pugh (C-P) scores. Model parameters were calculated using maximum-likelihood estimations. During-RT liver functional maps were predicted from dose distributions and pre-RT k1 maps with a conditional Wasserstein generative adversarial network (cWGAN). Imaging prediction quality was assessed using root-mean-square error (RMSE) and structural similarity (SSIM) metrics. The dose-response and NTCP were fit on both original and cWGAN predicted images and compared using a Wilcoxon signed-rank test. RESULTS: Logistic dose response models for changes in k1 yielded D50 of 35.2 (95% CI: 26.7-47.5) Gy and k of 0.62 (0.49-0.75) for the whole population. The high baseline ALBI (poor liver function) subgroup showed a significantly smaller D50 of 11.7 (CI: 9.06-15.4) Gy and larger k of 0.96 (CI: 0.74-1.22) compared to a low baseline ALBI (good liver function) subgroup of 54.8 (CI: 38.3-79.1) Gy and 0.59 (CI: 0.48-0.74), with p-values of < 0.001 and = 0.008, respectively, which indicates higher radiosensitivity for the worse baseline liver function cohort. Subset analyses were also performed for high/low baseline CP subgroups. The corresponding NTCP models showed good agreement for the fit parameters between cWGAN predicted and the ground-truth during-RT images with no statistical differences for low ALBI subgroup. CONCLUSIONS: NTCP models which incorporate voxel-wise functional information from DGAE-MRI k1 maps were successfully developed and feasibility was demonstrated in a small patient cohort. cWGAN predicted functional maps show promise for estimating localized patient-specific response to RT and warrant further validation in a larger patient cohort.

The Effect of Stereotactic Body Radiation Therapy for Hepatocellular Cancer on Regional Hepatic Liver Function.

PURPOSE: To investigate direct radiation dose-related and inflammation-mediated regional hepatic function losses after stereotactic body radiation therapy (SBRT) in patients with hepatocellular carcinoma (HCC) and poor liver function. METHODS AND MATERIALS: Twenty-four patients with HCC enrolled on an IRB-approved adaptive SBRT trial had liver dynamic gadoxetic acid-enhanced magnetic resonance imaging and blood sample collections before and 1 month after SBRT. Gadoxetic acid uptake rate (k1) maps were quantified for regional hepatic function and coregistered to both 2-Gy equivalent dose and physical dose distributions. Regional k1 loss patterns from before to after SBRT were analyzed for effects of dose and patient using a mixed-effects model and logistic function and were associated with pretherapy liver-function albumin-bilirubin scores. Plasma levels of tumor necrosis factor α receptor 1 (TNFR1), an inflammation marker, were correlated with mean k1 losses in the lowest dose regions by Spearman rank correlation. RESULTS: The whole group had a k1 loss rate of 0.4%/Gy (2-Gy equivalent dose); however, there was a significant random effect of patient in the mixed-effect model (P < .05). Patients with poor and good liver functions lost 50% of k1 values at 12.5 and 57.2 Gy and 33% and 16% of k1 values at the lowest dose regions (<5 Gy), respectively. The k1 losses at the lowest dose regions of individual patients were significantly correlated with their TNFR1 levels after SBRT (P < .02). CONCLUSIONS: The findings suggest that regional hepatic function losses after SBRT in patients with HCC include both direct radiation dose-dependent and inflammation-mediated effects, which could influence how to manage these patients to preserve their liver function after SBRT.

Current status and future developments in predicting outcomes in radiation oncology.

Advancements in data-driven technologies and the inclusion of information-rich multiomics features have significantly improved the performance of outcomes modeling in radiation oncology. For this current trend to be sustainable, challenges related to robust data modeling such as small sample size, low size to feature ratio, noisy data, as well as issues related to algorithmic modeling such as complexity, uncertainty, and interpretability, need to be mitigated if not resolved. Emerging computational technologies and new paradigms such as federated learning, human-in-the-loop, quantum computing, and novel interpretability methods show great potential in overcoming these challenges and bridging the gap towards precision outcome modeling in radiotherapy. Examples of these promising technologies will be presented and their potential role in improving outcome modeling will be discussed.

A systematic review and quality of reporting checklist for repeatability and reproducibility of radiomic features.

PURPOSE: Although quantitative image biomarkers (radiomics) show promising value for cancer diagnosis, prognosis, and treatment assessment, these biomarkers still lack reproducibility. In this systematic review, we aimed to assess the progress in radiomics reproducibility and repeatability in the recent years. METHODS AND MATERIALS: Four hundred fifty-one abstracts were retrieved according to the original PubMed search pattern with the publication dates ranging from 2017/05/01 to 2020/12/01. Each abstract including the keywords was independently screened by four observers. Forty-two full-text articles were selected for further analysis. Patient population data, radiomic feature classes, feature extraction software, image preprocessing, and reproducibility results were extracted from each article. To support the community with a standardized reporting strategy, we propose a specific reporting checklist to evaluate the feasibility to reproduce each study. RESULTS: Many studies continue to under-report essential reproducibility information: all but one clinical and all but two phantom studies missed to report at least one important item reporting image acquisition. The studies included in this review indicate that all radiomic features are sensitive to image acquisition, reconstruction, tumor segmentation, and interpolation. However, the amount of sensitivity is feature dependent, for instance, textural features were, in general, less robust than statistical features. CONCLUSIONS: Radiomics repeatability, reproducibility, and reporting quality can substantially be improved regarding feature extraction software and settings, image preprocessing and acquisition, cutoff values for stable feature selection. Our proposed radiomics reporting checklist can serve to simplify and improve the reporting and, eventually, guarantee the possibility to fully replicate and validate radiomic studies.

A deep survival interpretable radiomics model of hepatocellular carcinoma patients.

This work aims to identify a new radiomics signature using imaging phenotypes and clinical variables for risk prediction of overall survival (OS) in hepatocellular carcinoma (HCC) patients treated with stereotactic body radiation therapy (SBRT). 167 patients were retrospectively analyzed with repeated nested cross-validation to mitigate overfitting issues. 56 radiomic features were extracted from pre-treatment contrast-enhanced (CE) CT images. 37 clinical factors were obtained from patients' electronic records. Variational autoencoders (VAE) based survival models were designed for radiomics and clinical features and a convolutional neural network (CNN) survival model was used for the CECT. Finally, radiomics, clinical and raw image deep learning network (DNN) models were combined to predict the risk probability for OS. The final models yielded c-indices of 0.579 (95%CI: 0.544-0.621), 0.629 (95%CI: 0.601-0.643), 0.581 (95%CI: 0.553-0.613) and 0.650 (95%CI: 0.635-0.683) for radiomics, clinical, image input and combined models on nested cross validation scheme, respectively. Integrated gradients method was used to interpret the trained models. Our interpretability analysis of the DNN showed that the top ranked features were clinical liver function and liver exclusive of tumor radiomics features, which suggests a prominent role of side effects and toxicities in liver outside the tumor region in determining the survival rate of these patients. In summary, novel deep radiomic analysis provides improved performance for risk assessment of HCC prognosis compared with Cox survival models and may facilitate stratification of HCC patients and personalization of their treatment strategies. Liver function was found to contribute most to the OS for these HCC patients and radiomics can aid in their management.

Artificial Intelligence for Response Evaluation With PET/CT.

Positron emission tomography (PET)/computed tomography (CT) are nuclear diagnostic imaging modalities that are routinely deployed for cancer staging and monitoring. They hold the advantage of detecting disease related biochemical and physiologic abnormalities in advance of anatomical changes, thus widely used for staging of disease progression, identification of the treatment gross tumor volume, monitoring of disease, as well as prediction of outcomes and personalization of treatment regimens. Among the arsenal of different functional imaging modalities, nuclear imaging has benefited from early adoption of quantitative image analysis starting from simple standard uptake value normalization to more advanced extraction of complex imaging uptake patterns; thanks to application of sophisticated image processing and machine learning algorithms. In this review, we discuss the application of image processing and machine/deep learning techniques to PET/CT imaging with special focus on the oncological radiotherapy domain as a case study and draw examples from our work and others to highlight current status and future potentials.

Tumor response prediction in 90Y radioembolization with PET-based radiomics features and absorbed dose metrics.

PURPOSE: To evaluate whether lesion radiomics features and absorbed dose metrics extracted from post-therapy 90Y PET can be integrated to better predict outcomes in microsphere radioembolization of liver malignancies METHODS: Given the noisy nature of 90Y PET, first, a liver phantom study with repeated acquisitions and varying reconstruction parameters was used to identify a subset of robust radiomics features for the patient analysis. In 36 radioembolization procedures, 90Y PET/CT was performed within a couple of hours to extract 46 radiomics features and estimate absorbed dose in 105 primary and metastatic liver lesions. Robust radiomics modeling was based on bootstrapped multivariate logistic regression with shrinkage regularization (LASSO) and Cox regression with LASSO. Nested cross-validation and bootstrap resampling were used for optimal parameter/feature selection and for guarding against overfitting risks. Spearman rank correlation was used to analyze feature associations. Area under the receiver-operating characteristics curve (AUC) was used for lesion response (at first follow-up) analysis while Kaplan-Meier plots and c-index were used to assess progression model performance. Models with absorbed dose only, radiomics only, and combined models were developed to predict lesion outcome. RESULTS: The phantom study identified 15/46 reproducible and robust radiomics features that were subsequently used in the patient models. A lesion response model with zone percentage (ZP) and mean absorbed dose achieved an AUC of 0.729 (95% CI 0.702-0.758), and a progression model with zone size nonuniformity (ZSN) and absorbed dose achieved a c-index of 0.803 (95% CI 0.790-0.815) on nested cross-validation (CV). Although the combined models outperformed the radiomics only and absorbed dose only models, statistical significance was not achieved with the current limited data set to establish expected superiority. CONCLUSION: We have developed new lesion-level response and progression models using textural radiomics features, derived from 90Y PET combined with mean absorbed dose for predicting outcome in radioembolization. These encouraging, but limited results, will need further validation in independent and larger datasets prior to any clinical adoption.

Machine and deep learning methods for radiomics.

Radiomics is an emerging area in quantitative image analysis that aims to relate large-scale extracted imaging information to clinical and biological endpoints. The development of quantitative imaging methods along with machine learning has enabled the opportunity to move data science research towards translation for more personalized cancer treatments. Accumulating evidence has indeed demonstrated that noninvasive advanced imaging analytics, that is, radiomics, can reveal key components of tumor phenotype for multiple three-dimensional lesions at multiple time points over and beyond the course of treatment. These developments in the use of CT, PET, US, and MR imaging could augment patient stratification and prognostication buttressing emerging targeted therapeutic approaches. In recent years, deep learning architectures have demonstrated their tremendous potential for image segmentation, reconstruction, recognition, and classification. Many powerful open-source and commercial platforms are currently available to embark in new research areas of radiomics. Quantitative imaging research, however, is complex and key statistical principles should be followed to realize its full potential. The field of radiomics, in particular, requires a renewed focus on optimal study design/reporting practices and standardization of image acquisition, feature calculation, and rigorous statistical analysis for the field to move forward. In this article, the role of machine and deep learning as a major computational vehicle for advanced model building of radiomics-based signatures or classifiers, and diverse clinical applications, working principles, research opportunities, and available computational platforms for radiomics will be reviewed with examples drawn primarily from oncology. We also address issues related to common applications in medical physics, such as standardization, feature extraction, model building, and validation.

Machine Learning and Imaging Informatics in Oncology.

In the era of personalized and precision medicine, informatics technologies utilizing machine learning (ML) and quantitative imaging are witnessing a rapidly increasing role in medicine in general and in oncology in particular. This expanding role ranges from computer-aided diagnosis to decision support of treatments with the potential to transform the current landscape of cancer management. In this review, we aim to provide an overview of ML methodologies and imaging informatics techniques and their recent application in modern oncology. We will review example applications of ML in oncology from the literature, identify current challenges and highlight future potentials.

Machine learning for radiomics-based multimodality and multiparametric modeling.

Due to the recent developments of both hardware and software technologies, multimodality medical imaging techniques have been increasingly applied in clinical practice and research studies. Previously, the application of multimodality imaging in oncology has been mainly related to combining anatomical and functional imaging to improve diagnostic specificity and/or target definition, such as positron emission tomography/computed tomography (PET/CT) and single-photon emission CT (SPECT)/CT. More recently, the fusion of various images, such as multiparametric magnetic resonance imaging (MRI) sequences, different PET tracer images, PET/MRI, has become more prevalent, which has enabled more comprehensive characterization of the tumor phenotype. In order to take advantage of these valuable multimodal data for clinical decision making using radiomics, we present two ways to implement the multimodal image analysis, namely radiomic (handcrafted feature) based and deep learning (machine learned feature) based methods. Applying advanced machine (deep) learning algorithms across multimodality images have shown better results compared with single modality modeling for prognostic and/or prediction of clinical outcomes. This holds great potentials for providing more personalized treatment for patients and achieve better outcomes.

Automatic recognition and analysis of metal streak artifacts in head and neck computed tomography for radiomics modeling.

BACKGROUND AND PURPOSE: Computed tomography (CT) radiomics of head and neck cancer (HNC) images is susceptible to dental implant artifacts. This work devised and validated an automated algorithm to detect CT metal artifacts and investigate their impact on subsequent radiomics analyses. A new method based on features from total variation, gradient directional distribution, and Hough transform was developed and evaluated. MATERIALS AND METHODS: Two HNC datasets were analyzed: a training set of 131 patients for developing the detection algorithm and a testing set of 220 patients. Seven designated features were extracted from ROIs (regions of interest) and machine learning with random forests was used for building the artifact detection algorithm. Performance was assessed using the area under the receiver operating characteristics curve (AUC). RESULTS: The testing results of artifacts detection yielded a cross-validated AUC of 0.91 (95% CI: 0.89-0.94), and a test AUC of 0.89. External testing validation yielded an accuracy of 0.82. For radiomics model prediction, training with artifacts yielded an AUC of 0.64 (95% CI: 0.63-0.65), while training on images without artifacts improved the AUC to 0.75 (95% CI: 0.74-0.76). This was compared to visual inspection of artifacts (AUC = 0.71 [95% CI: 0.69-0.73]). CONCLUSION: We developed a new method for automated and efficient detection of streak artifacts. We also showed that such streak artifacts in HNC CT images can worsen the performance of radiomics modeling.

Balancing accuracy and interpretability of machine learning approaches for radiation treatment outcomes modeling.

Radiation outcomes prediction (ROP) plays an important role in personalized prescription and adaptive radiotherapy. A clinical decision may not only depend on an accurate radiation outcomes' prediction, but also needs to be made based on an informed understanding of the relationship among patients' characteristics, radiation response and treatment plans. As more patients' biophysical information become available, machine learning (ML) techniques will have a great potential for improving ROP. Creating explainable ML methods is an ultimate task for clinical practice but remains a challenging one. Towards complete explainability, the interpretability of ML approaches needs to be first explored. Hence, this review focuses on the application of ML techniques for clinical adoption in radiation oncology by balancing accuracy with interpretability of the predictive model of interest. An ML algorithm can be generally classified into an interpretable (IP) or non-interpretable (NIP) ("black box") technique. While the former may provide a clearer explanation to aid clinical decision-making, its prediction performance is generally outperformed by the latter. Therefore, great efforts and resources have been dedicated towards balancing the accuracy and the interpretability of ML approaches in ROP, but more still needs to be done. In this review, current progress to increase the accuracy for IP ML approaches is introduced, and major trends to improve the interpretability and alleviate the "black box" stigma of ML in radiation outcomes modeling are summarized. Efforts to integrate IP and NIP ML approaches to produce predictive models with higher accuracy and interpretability for ROP are also discussed.

Radiomics in precision medicine for lung cancer.

With the improvement of external radiotherapy delivery accuracy, such as intensity-modulated and stereotactic body radiation therapy, radiation oncology has recently entered in the era of precision medicine. Despite these precise irradiation modalities, lung cancers remain one of the most aggressive human cancers worldwide, possibly because of diverse genotypic alterations that drive and maintain lung tumorigenesis. It has been long recognized that imaging could aid in the diagnosis, tumor delineation, and monitoring of lung cancer. Moreover, accumulating evidence suggests that imaging information could be further used to tailor treatment type and intensity, as well as predict treatment outcomes in radiotherapy. However, these imaging tasks have been carried out either qualitatively or using simplistic metrics that doesn't take advantage of the full scale of imaging knowledge. Radiomics, which is a recent field of research that aims to provide a more quantitative representation of imaging information relating tumor phenotypes to clinical and genotypic endpoints by embedding extracted image features into predictive mathematical models. These predictive models can be a key component in the clinician decision making and treatment personalization. This review provides an overview of the radiomics application and its methodology for radiation oncology studies in lung cancer.